Judging the judges: evaluating the accuracy and national bias of international gymnastics judges

Abstract We design, describe and implement a statistical engine to analyze the performance of gymnastics judges with three objectives: (1) provide constructive feedback judges, executive committees national federations; (2) assign best most important competitions; (3) detect bias persistent misjudging. Judging routine is random process, we model this process using heteroscedastic variables. The developed marking score scales difference between mark judge true level gymnast as function intrinsic judging error variability estimated from historical data for each apparatus. This dependence quality has never been properly studied. leverage outlier marks study favoring athletes same nationality. also ranking scores assessing what extent rate gymnasts in correct order. Our main observation that there are significant differences worst both terms accuracy bias. insights work have led recommendations rule changes at Fédération Internationale de Gymnastique.